Best Arm Identification in Restless Markov Multi-Armed Bandits

We study the problem of identifying best arm in a multi-armed bandit environment when each is time-homogeneous and ergodic discrete-time Markov process on common, finite state space. The evolution governed by arm’s transition probability matrix (TPM). A decision entity that knows set TPMs but not exact mapping to arms, wishes find index as quickly possible, subject an upper bound error probability. selects one at time sequentially, all unselected arms continue undergo (restless arms). For this problem, we derive first-known instance-dependent asymptotic lower growth rate expected required arm, where asymptotics vanishes. Further, propose sequential policy that, for input parameter $R$ , forcibly has been selected consecutive instants. show achieves depends monotonically non-increasing notation="LaTeX">$R\to \infty $ . question whether, general, limiting value matches with bound, remains open. identify special case which bounds match. Prior works identification have dealt (a) independent identically distributed observations from (b) rested whereas our work deals more difficult setting restless arms.